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ABSTRACT 

Needs assessment in legislative libraries has three critical 
components: a well-thought out strategy to identify what should be understood 
about Members' research and information needs, expectations and preferences; 
a range of reliable, effective and appropriate methods to find this critical 
information; and ways of building what is learned into all stages of 
strategic planning and service development. This paper first provides an 
introduction which discusses the definition of and rationale for needs 
assessment. It then outlines the following eight ways of getting critical 
information on Members' needs, expectations and service use: (1) Capture 

anecdotal information from Members wherever it can be found; (2) More 
proactively, ask for feedback on your services - and make it easy; (3) Look 
for systematic ways to canvass clients; (4) Pilot test all new service or 
product initiatives; (5) Look for captive audiences; (6) Regularly check in 
with key stakeholders; (7) Do intensive interviews to answer specific 
questions; and (8) Be as comprehensive as resources allow. Recommendations 
for putting client insights into action conclude the paper. (AEF) 
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Introduction 



Why bother about client needs assessment research - aren’t we in the business of providing direct 
services? In fact, I think understanding client needs is the essential foundation of good service delivery. 
We all have broadly similar mandates of providing research and information services to help our Members 
and legislatures do their work. And we all see ourselves as client driven. This means we need to know 
what services our clients need and how well the services we are providing actually meet those needs. 

Is needs assessment something that only the richer parliaments can afford? Are comprehensive studies 
and consultants a frill or luxury beyond most of us? While we all have to live within our means, I think 
there is a wide range of effective needs assessment we can do within limited resources. 

Before outlining some of these methods, here are a few starting points: 



What is Needs Assessment? 

• I think needs assessment has three critical components: 

. a well-thought out strategy to identify what we need to understand about our Members’ 
research and information needs, expectations and preferences; 

. a range of reliable, effective and appropriate methods to find this critical information; and 

. just as importantly, ways of building what we learn into all stages of our strategic 
planning and service development. 

• There are really two separate issues here: assessing what services and products Members need, 
and evaluating how well our services are actually meeting those needs. The balance will vary: if 
we are considering a new service, then we need to assess how strong the need for it is; if we are 
reviewing established research services, then we need to evaluate what Members think of the 
services and how effectively we are providing them. In practice, research and feedback from 
clients can serve both assessment and evaluation purposes. 

• Our methods must be: 

• reliable - yielding information on our clients that is sound and useful, that we can 
confidently act on; 

• effective — so that we don’t have to devote too much of our scarce resources to finding out 
this information; and 

• appropriate - geared to our unique political environments; absolutely confidential and 
not intrusive on our busy Members. 

• I will be emphasizing a wide range or continuum of methods. 1 think we never want to rely 
entirely on one or a few methods. The real value of such a continuum is as a whole; by 
researching client needs and use in different ways and getting feedback from different sources, we 
gradually build up a sounder overall picture. It also allows us to adapt those methods that best suit 
our resources and situations. 

Be Prepared to Act 

• We should only do needs assessment research or ask for feedback from Members if we are 
serious. We have to be prepared to act on what we learn. We may very well be raising their 
expectations by simply going to them. 



• And that means that we have to organize ourselves to be able to feed the knowledge we gain from 
clients into our service planning. 

• I like to think of continuous planning loops: research to understand what Members need -a- 
which guides our strategic planning and priorities which guides which services we develop and 
how —> we then evaluate how well the services are working from the clients’ point of view + 
monitor ongoing client feedback -a- to guide continuous refinement and improvement of the range 
of services we provide. 

Start from Strategic Priorities 

• We need to start from our strategic situation -> that determines what we need to know 

• for example, if you are going to develop an Intranet, you need to know if your Members 
have the infrastructure and skills to benefit from electronic products and resources, and to 
determine which products to develop, you need to know what information they need 
most; 

. if you are planning to offer research services to committees for the first time, you need to 
know very concretely how your committees do their work, what their major priorities are 
(legislative drafting, financial oversight, accountability, policy development, etc.), what 
analysis they need to support these priorities and activities, what specific services are 
going to be most valuable to them, etc.; 

. new Research Services face the quite fundamental questions of what info and analysis do 
their Members need and what research do they provide to meet those needs? 

. on the other hand, in more established services, we think we understand our Members' 
information needs (however imperfectly). But we still need to check that our 
understandings are sound, and we need to know how our Members’ expectations and 
practices are changing. Plus, even if we think we understand their needs pretty well, the 
real challenge is to make sure we are actually meeting those needs. So established 
institutions may be focussing more on evaluating the services provided than assessing 
needs. 

• What we are also doing is separating out what we absolutely need to know to achieve our strategic 
priorities from what it would simply be nice to know. 



Range of Methods 

Once we have identified the key things about our Members we need to understand better, how do we go 
about finding out? How can we do effective needs assessment within limited resources? 

I am going to outline eight ways of getting critical information on Members’ needs, expectations and 
service use; some of these methods are both cheap and effective. I don’t mean this to sound like the kind 
of simplistic prescriptions we see in business writing (the six characteristics of effective presentations, the 
four keys to personal financial success ..); rather that these are some options to think about. I’ll give a few 
concrete examples of each; and other colleagues will be able to fill in many more. 

None of this is meant as a blueprint: we all have to adapt methods that work in our particular situations. 



These various types of needs assessment and service evaluation are set out roughly: 



• from less to more rigorous in methodological terms; 
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Needs Assessment 
Equation 

what you need to know + 
available resources (all in 
the context of your 
particular parliamentary 
environment) — > how you 
find out 



• from less to more costly or resource intensive; 

• but not necessarily from least to most effective. I think that effectiveness and 
reliability comes from pulling together the results of different methods, as 
well as from effective design and implementation of individual projects. 



1: Capture anecdotal information from Members wherever you find it 

• We all have a source of information on Members’ expectations and service use that is free and 
easy to find - talking to them and their staff. Part of our job as managers is keeping in touch with 
Members and their staff anyway; all we have to do is record what they are saying. 

• A related and easy method is to tap staff intelligence. Front-line staff are in constant touch with 
clients and over time develop a good overall sense of what clients are doing and thinking. For 
example, my Branch has recently had a series of front-line staff forums on such topics as client 
and staff use of our Intranet products and client misperceptions about our services. (I used to use a 
written questionnaire to canvass staff impressions and observations but semi-structured focus 
group-type settings have yielded far richer information.) 

• We might rightly hesitate to rely on such anecdotal information because it is not seen as rigorous 
or scientific. But I think that systematically collecting and correlating comments, suggestions and 
other tidbits from clients can be a rich source of insight on how they actually use our services and 
see us. 

• Interpreting and using anecdotal information is a methodological challenge: 

. the main danger is that the people you are hearing from are not representative of all your 
Members - especially if they are your most frequent users; 

. so we need to also find ways to hear from Members that don’t use us as much. A 

strategic priority may be to find out why and to correct any problems or barriers. This can 
be done through the various interviews and other means outlined later; 

. another danger is that anecdotal input is by its nature partial - only on the issue the client 
is interested in at the time as opposed to our overall range of services; 

. we also don’t want to over-react to every good idea or suggestion from individual 
Members - we are really interested in understanding patterns and seeing trends; 

• We certainly have to be aware of the methodological limits of what we hear from Members, but 
that does not mean that the information is not useful: 



• one major advantage of open-ended conversations with clients is that they may tell us 
things we didn’t know that we didn’t know, or don’t want to hear, or never would have 
imagined for ourselves; 

• so, listening to Members can be a source of tremendously rich insight on how they 
perceive our services, but it is only one source among many and we do need to test these 
insights against more rigorous methods. 

2: More proactively, ask for feedback on your services — and make it easy 

• Our friends at Parliamentary Research in Ottawa include short response forms in every project 
they send out to Members. The response rate is a very good 45-50% and the results provide an 
ongoing indicator of overall satisfaction and many specific comments. 

• My Library has feedback buttons all over our Intranet site and on specific electronic products. 

We occasionally put short questionnaires up: e.g. are clients using a new search engine we put on 
a Bills product? are they finding it useful? any suggestions for better presentation? The response 
rate has not been high, we think because clients are so busy and want to use the electronic 
products quickly and get out again. 

• Many parliamentary libraries have comments, complaints or suggestion boxes. 

• I personally think that having these feedback opportunities widely available is important even if 
there is not a high response rate and they do not yield that much useful information. We can 
constantly emphasize to clients that we are really interested in hearing from them, especially on 
gaps they see in our services and any suggestions for improvements. 

• Hearing from lots of different clients in these many ways - even if still anecdotal and unstructured 
- can reveal consistent patterns over time. But to identify these patterns we need some systematic 
tools and processes: 

. we need to record the feedback we get from clients and our observations on clients in an 
organized way; 

. we then need to analyze these many bits of information to spot patterns and connections 

. this collection and correlation can be much easier if you set up a database, which can also 

link to results of more structured research methods to be discussed next. Australia has a 
‘bouquets and brickbats’ database; 

. but if that isn’t possible, we can all make notes, review them regularly (which, of course, 
is also the trick with a database - the most sophisticated databases are useless if we don’t 
review the results), make reporting feedback from Members a routine part of our staff 
meetings (when my branch has a go-round of current client questions, I want to hear not 
just what the questions were, but what the clients said about how the project worked out). 

3: Look for systematic ways to canvass clients 

• To get beyond the methodological limitations of anecdotal information, we need to conduct more 
systematic research with clients. 

• My Library has developed an example of small-scale structured interviews; what we call contact 
point research. We wanted to take advantage of the place where we most often interact with 
clients - when they call in questions to our phone reference desk. 

. after the client makes their request, we ask them if they can spare us a minute. We then 
pose one or two simple and structured questions (we have a supply of question sheets at 
the reference desk and the librarians simply record the responses on the sheet). 
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. most clients were happy to answer the few questions. Those who were too busy didn’t 
seem to resent being asked and often said they would answer next time they called. 

• This can be an easy and non-intrusive way of getting a significant number of responses fairly 
quickly from a reasonable sample of users (which remember, may not be the same as all 
Members). For example, we have used this method to estimate use of our Intranet site and 
products. By repeating the same process every few years, we can identify trends. We also record 
impromptu remarks and have gathered quite a number of useful suggestions for product 
improvements. 

• This is a method that could be adapted by any service, even those with very few resources. But 
not for complex issues: because we are using a busy service point and busy front-line staff, we can 
ask only a few straight-forward questions. 

• There is another source of systematic information that we all keep - service statistics. Stats are 
good ways of spotting trends in demand for services (if particular Members stopped using our 
services, we would want to explore why), for measuring client activity in more quantitative areas 
such as hits on Intranet products (if one product was used 50 X more than another), and for 
helping us determine where to concentrate our efforts. 

. There is a danger in collecting stats on everything we do or just because we always have. 
Stats are only as good as the planning that goes into deciding what to collect and for what 
purposes. We only want those stats that help us plan and monitor service delivery. And 
we don’t want to have to put huge amounts of time into collecting them. 

. Where they can be very useful is as part of strategic planning: we identify our top 

priorities for the next few years, devise indicators to show how successful we are, develop 
stats to measure these indicators, and use them to evaluate how well we are doing against 
our priorities. 

4: Pilot test all new service or product initiatives 

• Pilot testing can be a cost-effective method of yielding actionable information, is absolutely 
essential to service planning and development, and can also be done on a small scale. 

• Whenever we are considering or implementing a new service we need to know: Is the new service 
going to be of value to clients? More specifically, is it going to be valuable enough to justify our 
allocating staff and other resources? Once rolled out, are Members using the new service the way 
we anticipated? Are there changes we need to make to the pilot version of the services to better 
meet client needs? 

• We can go to clients to assess if there is a gap in our services which the potential new service is 
intended to fill. But the main danger of simply asking clients about a new service is that they are 
likely to say yes uncritically, especially if it is something entirely new that they find hard to 
visualize; 

. one technique is to present clients with a prototype of the potential new service and ask 
them if and how they would use it; 

. another is to pose our usual service dilemma to them starkly: “ all right, you will use this 
new service X, but to do X what should we drop - Y or Z ?” 

• An electronic variant is usability research. This involves sitting clients down in front of a 
computer and watching them actually use our site and products (e.g. we ask them to find certain 
information and observe how they go about it, or how they move about within the structure of a 
particular product). This shows us how clients actually use the products, as opposed to how we 
thought (or hoped) they would in our planning. It is absolutely crucial to fine-tuning our site and 




9 



product design and making sure that the products really are useful to clients. This methodology 
requires quite small samples and sessions of less than an hour, so doing usability research need 
not involve a huge investment in time. 1 think it should be an essential part of any development of 
electronic products. 

• We can also follow-up new service initiatives. For example, my Library has been providing more 
value-added services from our librarians: rather than just a package of key articles and reports, a 
memo synthesizing the material; rather than copied pages of financial data from various years, a 
table highlighting the key trends; rather than a list of Internet resources, a memo sent 
electronically so that the links are hot and the client just has to click to get to them. We will be 
calling a sample of Members’ offices who have received such projects to ask how useful the 
value-added features were to them. In other words, we think these enhancements make sense, but 
we need to see how valuable they are in the clients’ own terms, how well their particular purposes 
are met. 

• All of this requires us to be hard-headed as managers. We have to be prepared to cancel what we 
thought (or hoped) were going to be good initiatives if they aren’t being used by clients or aren’t 
seen as valuable enough to justify the resources put into them. 

5: Look for captive audiences 

• By this I mean people who are around the legislature and easy to interview, and who are either 
major users themselves or know about Members’ use. To be blunt, these are people who feel 
obliged to help the Library out. 

• Ontario, like other parliaments, has an interns program in which recent graduates work in the 
Assembly for a year to experience the legislative environment and enhance their skills. We 
interview them every spring, with a mixture of general questions to help us understand Members’ 
overall use of Library services, and more focussed questions on specific issues we may be 
considering at the time. The interns have several critical advantages as a source of information: 
they alternate a term with a government and a term with an opposition Member, so that the 8 
interns can tell us about 16 client offices; they do significant policy and legislative work in these 
offices, which we are particularly interested in; they are themselves sophisticated information and 
research users (partially because we train them), so they are good examples of more high-end 
users, especially of electronic products; and because we have now interviewed them for five 
years, they have become an invaluable source for historical comparison. It is also fairly easy to 
impose on them because one of our Research Officers co-ordinates the program. 

• Speakers, Clerks or public relations offices in our legislatures often ask for background papers for 
Members attending parliamentary or other conferences, on jurisdictions Members are visiting, on 
the home countries of visiting delegations, or on other legislative issues. We can ask the 
colleague who co-ordinated the conference or visit how the papers worked out. 

6: Regularly check in with key stakeholders 

• I interview committee chairs at least once in the life of each parliament. My goal is to ensure 
they are happy with the services we provide them and to see if there is anything they would like to 
change or improve. I always take the opportunity to ask them what they think of Library research 
and information services and for their general suggestions for enhancements; adding to our overall 
picture from another 15 or so key Members. 

• In our Assembly each party has its own research staff. I tend to meet with their heads each 
summer in the quieter period between sessions. As with the interns, the caucus researchers have 
worked very closely with the Library over the years and it is in their interests to let us know what 
they need. As with the Committee chairs, I want to check that our services are meeting their 
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particular needs - generally very demanding, with tight deadlines, and intensive use of 
subscription databases and other electronic resources. They are also a good general source on how 
Members from their caucus are using Library services and have had good ideas over the years for 
service improvements. 

• Exit interviews with experienced staff. A methodological problem with any interviews is that if 
you are seen as nice and responsive (and all researchers are!), then clients sometimes may not 
want to hurt your feelings by seeming to complain. An advantage here is that people leaving can 
sometimes be more frank. 

7: Do intensive interviews to answer specific questions 

• One critical means of more systematic research which can yield extremely valuable results is 
personal interviews. To be effective, we have to consider a sample that is large enough and 
balanced enough (this will vary for our different parliaments, but generally includes by party, 
region, experience, offices held within the government or legislature, etc.) and with a question 
framework that will yield reliable responses. Interviews can be informal or more structured, use 
fixed questions or be more open-ended, and can focus on specific issues or be designed to yield 
broader information on clients’ overall service use, expectations and perceptions. 

• Interviews with a reasonable sample and tightly focussed questions can be used to answer specific 
operational or planning questions. These can be short - 20 minutes or so - and can be done over 
the phone if we can’t pin busy Members down. 

• One example we did several years ago involved newly elected Members. An election was 
expected within the year and we were beginning to plan our client relations campaign to the new 
House. We wanted to know what kinds of information and orientation rookie Members needed. 
We interviewed a sample of Members first elected to the current parliament about their experience 
when they arrived. This yielded practical and rich info which we used to guide our planning for 
the next cohort of new Members; for example, they said the first few weeks were incredibly 
hectic, they were bombarded with paper, and suffered from information overload; so we 
concentrated more on personal visits after they were settled into their offices. 

• Sometimes the research is, in effect, commissioned by other branches or units within the Library. 
For example, colleagues in our Information Resources branch were considering enhancing our 
electronic press clippings. Did Members want more newspapers included or to have the database 
go back more than the current year; either or both would require increased licensing costs. We 
interviewed a sample of Members and staff and found that while both options would be nice, they 
did not value them that highly. We decided not to enhance the database in these directions and 
saved a considerable amount that was allocated to other resources. In these ways, client research 
can be used as a key decision-making tool by managers throughout the Library. 

• I haven’t been mentioning the traditional written questionnaire. Probably most of our libraries 
have mailed out a standardized and mostly quantitative set of questions in which clients are asked 
to tick off pre-determined choices or rate services on a scale. I actually don’t think such highly 
structured surveys yield the nuanced information we need. 

• But written fixed choice questionnaires can be useful for specifically focussed issues. For 
example, we use short questions at the end of client training sessions to evaluate how 
useful the training was and identify further areas clients want covered. 

• Short questionnaires can also be a useful starting point in sunset reviews of services. We 
have recently sent questionnaires out with a paper status of legislation product. We want 
to discontinue the print version because we have a far richer electronic product on the 
Intranet and it takes considerable redundant time to produce, copy and distribute, time we 



would like to re-allocate to other services. We gave Members several yes/no choices on 
discontinuing and asked them to write in any specific things they like about the print 
product or reasons why we shouldn’t scrap it. We are getting enough responses favouring 
the paper product that we may have to keep it. But we may change it to a subscription 
service (at the moment it goes to all Members automatically) to save production 
resources. 

• This type of research highlights that when we ask clients for specific advice and direction, we had 
better be prepared to respond to what they say (sometimes even when we aren’t doing everything 
clients wanted). 

• We also want to use such specifically focussed research to build up our bigger picture of 
Members’ needs, information use and expectations. In the case above of investigating preferences 
for electronic press services, we were also able to obtain useful information on how clients were 
using our Intranet and other electronic resources more generally. In fact, it is valuable to end even 
very focussed interviews by asking Members and their staff if they have any overall comments, 
especially any suggestions on new services or things we should be doing differently. 

8: Be as comprehensive as resources allow 

• At the most intensive end of the continuum of assessment methods, a comprehensive set of in- 
depth interviews with a broadly representative sample of Members and their staff can be 
tremendously useful. It will yield more nuanced and reliable information than smaller-scale 
projects. It can serve as a benchmark on changing client use and expectations, and as a standard 
against which we measure our own performance. A great deal of research and literature has been 
developed on public sector service quality research that can help us plan such projects. 

• We like to do a series of such interviews in the mid to late period of each Parliament. 

• This is the time to get in outside consultants if you can - highlighting another methodological 
issue. It is not appropriate for managers responsible for particular services to be evaluating how 
well they are being delivered; clients will not perceive them as objective and may not be frank 
with them. External consultants may also have interviewing and analytical skills that you don’t 
have in-house. 

• There is another methodological - or rather interpretive - issue that even the most sophisticated 
research must take into account. We can get reliable information on what Members and their staff 
want; that is, what they think they need. But that is not always the same as what they actually 
need. We are the research and information professionals, and part of the key value we add is in 
interpreting their requests and preferences, and using our expertise to help them understand what 
are the best research tools and strategies to serve their purposes. 

• Not everyone is going to be able to afford the extensive projects our Australian colleagues 
undertake. But your interviews can be geared to the resources you have; even a week of a 
consultant’s time to do 20 or so client interviews can be an investment that will yield 
tremendously valuable information. Another option is to have a trained Library staffer do the 
interviews, as long as they are not involved in direct service provision. 
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Conclusions: Client Insight Into action 



What do we need to be able to do all this? 

• First of all, a strategy: it is wasteful and ineffective - let alone irritating to busy Members - to 
have all kinds of people all over the Library launching their own client needs research. All needs 
assessment projects have to be well-focussed and co-ordinated. 

. Part of the strategy is also communications to Members. It is always useful to follow-up 
with the participants in projects to tell them what overall results were found and what we 
are going to do about it. Every time we follow-up in this way, we are reinforcing the 
theme that we do listen and respond to clients. 

. More generally, if you have a regular communications vehicle to Members, it is worth 
keeping them up to date on projects, findings and action taken in response. 

• We also need the requisite research skills. They are not that arcane; many researchers or 
managers with social science training will have studied or conducted such research. External 
training can also be a good investment. 

• Returning to the beginning, we need the commitment to act on the knowledge we gain about our 
clients’ needs, expectations and use patterns, and actually build that into our service planning and 
delivery. 

• And finally, we need processes and structures to translate the knowledge we gain on clients into 
sound strategic and operational planning. Key facets of this are: 

. A team or responsible manager to co-ordinate and implement the various needs 

assessment projects. My Library has a cross-branch Client Relations Team to play this 
strategic and co-ordinating role. It also acts as the forum for prioritizing and dovetailing 
the various issues that could be investigated and as a clearing house where managers or 
staff can bring issues and ideas they think should be addressed. 

. We also have a designated co-ordinator to keep track of the various projects and conduct a 
good number of them directly; these are part-time responsibilities combined with other 
parts of a person’s job. 

. We need clear and consistent planning loops to feed the knowledge gained to the 

managers or planning forums who need to know (plus expectations that all managers will 
respond to results relevant to their spheres). 

. We also need an appropriate use policy: even using just a few of these methods, plus 
keeping service stats, you will end up collecting a great deal of personal information on 
clients’ service use and opinions. Confidentiality is one of our core values, and must be 
absolutely guaranteed. But we also have to analyze the purposes for which we can use the 
information we collect, who should see what level of detail on client research and 
feedback, how long we keep the info, etc. 



We can be modest in our own expectations. We never get to make our strategic and operational decisions 
with all the information we would like. And we are never going to understand everything about our 
Members; they are too unpredictable and their environment changes too fast for that. But the more 
information we have on Members’ expectations, preferences, service use and satisfaction, the better we 
will be able to plan and refine our services to best meet their needs. 




13 



Deciding which methods to use is never an all-or-nothing proposition. If we don’t have the resources for 
comprehensive and systematic interviews, it doesn’t mean we do nothing. We can use the most effective 
methods possible at the time and within our resources. We save up those issues that are best handled in a 
comprehensive survey and we work away at issues that can be effectively done with small projects. 

We can also be flexible; adapting those methods that work best in our particular environment. And if a 
method didn’t work too well, then we can try another. I have emphasized seeing these different methods - 
and many others that other libraries have used - as a broad continuum from which we can draw the best 
methods to suit the situation. 

By seeing these methods as a continuum we can also think about how to effectively use them together: 

• these different methods can be used to test each other: 

. We may see a pattern in what we are hearing informally from Member’s staff and want to 
assess its validity with more systematic methods. For example, in our internal forums 
many front-line staff had heard clients say something along the lines of “we’re sorry to 
bother you .. we know you are so busy.” We are busy, of course, but we never want 
clients to be discouraged from calling. We will test to see how widespread and significant 
this perception is in a series of qualitative interviews in the fall. 

. Do hit stats on Intranet products confirm what staff are hearing on which products clients 
use the most? Are both of these sources consistent with what we hear directly from 
clients in interviews or contact-point research? These different sources did show the same 
broad patterns, and this helped us prioritize which products to enhance. 

. It is prudent to worry about the methodological rigour of individual projects, and we must 
be cautious about interpreting the results. But reliability and insight come not just from 
designing individual projects well, but from using a wide range of solid methods and 
investigating particular issues from several angles, all to build up a comprehensive overall 
picture of clients’ research and information needs. 

• the real challenge - and the real value - is in analyzing the results from all the various methods 
and projects as a whole, taking account of each’s methodological limitations and specific focus, 
but looking for common patterns and insights. I think we build up our understanding of 
Members’ needs through a continual process of learning about them in different ways, and testing 
and re-testing our findings and conclusions. 



Finally, to return again to the point about acting on whatever needs assessment we do: one of our scarcest 
resources is management ‘thinking time’. We need to somehow build in the time to reflect on the results 
and incorporate them into our planning and service delivery. Making effective use of what we have 
learned is the whole point. 
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